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Introduction and summary 


Students from low-income backgrounds face a variety of social and economic 
challenges that make it more difficult for them to achieve their potential. They 
often have challenging home lives. They are more likely to have health issues. 1 
They are exposed to millions fewer words than their more affluent peers and often 
lack access to high-quality early childhood education programs that could help 
them catch up. 2 To make matters worse, low-income students often attend public 
schools that receive less funding than schools serving more affluent students. 3 If 
all that were not enough, low-income students also are taught disproportionately 
by the most inexperienced and out-of-field teachers. 4 

However, it is also clear that some states do a far better job of educating 
low-income students than others. According to the National Assessment of 
Educational Progress, or NAEP, there is a massive gap between the states with 
the highest-performing low-income students and the states with the lowest. 

For example, in eighth-grade math, students from a low-income background in 
Massachusetts scored 17 points higher than low-income students in Mississippi. 5 
Or think of it this way: Low-income students in Massachusetts are a full grade and 
a half ahead of low-income students in Mississippi in eighth-grade mathematics. 6 

The Center for American Progress wanted to better understand the role of stan- 
dards-based reform in promoting student outcomes, and to that end, we studied 
the most recent NAEP data. Given previous research, we believed that we might 
find a strong connection between standards-based reform and student outcomes. 

Because it can be hard to make clear connections between policy and outcomes, 
some of our analysis is anecdotal in nature. We also used more-empirical tools for 
our study, relying on a statistical approach known as a regression analysis to unpack 
the relationship between standards-based reform and student outcomes. For that 
part of our analysis, we looked specifically at the performance of low-income stu- 
dents on NAEP over time in relation to a state’s standards-based reform efforts, as 
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measured by the Education Counts database maintained by Education Week. 7 Since 
policy takes time to have an effect across a state, we measured the impacts of policy 
improvements on NAEP outcomes two years after the actual policy change was 
adopted, taking into account time for policy to be implemented. 

We believe our findings are particularly relevant given the most recent release of 
NAEP data, and many critics of standards-based reform have expressed concern 
that NAEP scores did not tick upward between 2013 and 2015. 8 Our research 
took a more historical look, examining NAEP data over the past decade, and our 
findings suggest that there is clear evidence that standards-based reform works, 
particularly when it comes to the needs of low-income students. 

Based on our analysis, we found that: 

• Over the past decade, many states that have not fully embraced standards- 
based reform have fallen behind, while states that have thoughtfully pushed 
standards have shown clear gains. From 2003 to 2015, many states that have 
been historically averse to standards-based reform have shown some of the low- 
est rates of growth in NAEP scores for low-income students. There are about a 
dozen states that have shown less than a 5-point gain for low-income students, 
on average, including Kansas, Iowa, Idaho, Montana, and North and South 
Dakota. Generally speaking, these states appear to have not embraced stan- 
dards-based reform fully. Iowa, for instance, was one of the last states to endorse 
academic standards, adopting them in 1997. 9 

In fact, two states — North Dakota and South Dakota — actually showed a 
decline in NAEP scores for low-income populations from 2003 to 2015, on 
average. In other words, these two states have actually regressed in terms of 
high-poverty student performance. 

In contrast, the District of Columbia, Tennessee, Massachusetts, and Florida 
have had some of the largest gains on NAEP for high-poverty students since 
2003, and these areas are at the top of a list of states that have posted more than 
a 10-point jump on NAEP over the past 12 years. 

While there is an important debate over the definition of standards-based reform — 
and this analysis is undoubtedly anecdotal and impressionistic — it appears clear that 
states that have not embraced the approach have shown less success, while more 
reform-oriented states have shown higher gains over the long term. 
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• Implementing standards-based reform significantly improved learning outcomes 
for low-income students in fourth-grade math and eighth-grade reading. According 
to our analysis, states typically saw a jump in outcomes due to standards-based 
reform from 2003 to 2013, and states’ standards-based reform effort accounts 
showed positive outcomes in elementary school math and middle school reading. 

We modeled our analysis on a 2006 study of the impact of standards-based reform 
on student achievement by researcher Christopher Swanson, which looked at 
the impact of reforms on NAEP up to that point. Similar to the CAP analysis, 
Swansons study relied on Education Week data and showed positive NAEP out- 
comes for states that implemented standards-based reform as measured by the 
Education Counts indicators. 10 Our analysis took a slightly different approach, 
accounting for fiscal equity as well as focusing specifically on low-income students. 

Like all regression analyses, variations in our model can shift the outcomes. 
Without including a fiscal measure, for instance, policy changes generally 
have a weaker relationship with changes in achievement. That being said, our 
empirical results, together with our more anecdotal results, make us confident 
in our overall findings. 

• States posting poor results are among those looking to leave the Common 
Core State Standards, or Common Core — a set of higher academic K-1 2 stan- 
dards in reading and math — which were developed and adopted by governors 
and chief state school officers in 2010. Oklahoma and South Carolina have 
both recently left the Common Core. 11 What is notable about this development 
is that these two states generally have low levels of achievement, and they score 
below the national average in almost every tested subject area and grade level. 
The states also have a long way to go when it comes to low-income students. In 
South Carolina, for instance, just 14 percent of low-income students are at grade 
level in middle school math. 12 In Oklahoma, just 13 percent of low-income 
students are doing grade-level work in middle school math. 13 

Given these findings, we believe that states should remain dedicated to standards- 
based reform. The Common Core is the most recent major policy initiative to 
advance the broader standards-based reform approach. Because of its potential to 
drive reforms that benefit many students, states should continue their commit- 
ment to the Common Core’s full implementation and aligned assessments. 
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A short history of the standards- 
based reform movement 


In many ways, standards-based reform began in 1983 with the release of the 
groundbreaking report, “A Nation at Risk: The Imperative for Educational 
Reform/’ published by the U.S. Department of Education. The report — written 
by a commission comprised of members representing both political parties, along 
with business and education leaders — condemned the U.S. public education 
system for inadequately preparing all students for the workforce, particularly low- 
income students and students of color. 14 

“The educational foundations of our society are presently being eroded by a ris- 
ing tide of mediocrity that threatens our very future as a Nation and a people,” 
the report warned. “If an unfriendly foreign power had attempted to impose on 
America the mediocre educational performance that exists today, we might well 
have viewed it as an act of war.” 

The report recommended significant change in five areas: standards; content; 
time; teachers; and leadership. Specifically, it recommended higher entrance 
standards for colleges and universities, more instruction time spent on core 
subjects, regular tests of mastery of content, a professionalized teaching work- 
force, and particular attention paid to the needs of disadvantaged groups such as 
low- in come students and students of color. 15 

In response to the report’s call to action, policymakers sought ways to 
improve schools across the United States, and many argued that setting clear 
academic standards for student learning would guide improvements across 
the whole education system. 

Over the next six years, momentum for standards-based reform built until 1 989, 
when President George H.W. Bush hosted an education summit in Charlottesville, 
Virginia, with the nation’s governors. There, governors from both parties worked 
with the Bush administration to establish and commit to six educational priorities, 
including that all students would become competent in challenging subject matter. 16 
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A few years later, President Bill Clinton signed the Goals 2000: Educate America 
Act, which expanded on those earlier aims, requiring every state to have grade- 
level subject area standards and to assess student progress against those standards 
at least once each in elementary, middle, and high school. 17 This act was the first 
time that states were required to have academic standards as a condition of receiv- 
ing federal funds. President Clinton also called for common national academic 
standards in math and reading that the states would adopt voluntarily. 

The national commitment to standards-based reform was reaffirmed in the next 
reauthorization of ESEA: the No Child Left Behind Act, or NCLB. This law 
increased the number of times states were required to assess student progress. 
Specifically, states had to test yearly in third through eighth grade and once in high 
school. 18 Furthermore, NCLB required the disaggregation of student test scores 
by student subgroup. This important policy highlighted student achievement gaps. 
Finally, NCLB required states and districts to intervene in schools that were not 
improving student achievement. 

NCLB, while well-intentioned, was not without its shortcomings. Some states 
did not do enough to help teachers boost instruction, and some schools became 
overly focused on test scores. Many states also administered tests that were made 
up only of multiple-choice items, despite the fact that these sorts of exams do not 
adequately measure what students should know and be able to do. 19 
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The Common Core and standards- 
based reform today 


Today, vast differences in educational opportunities remain. What’s more, stu- 
dents in many communities are performing well below grade level. And while 
standards-based reform has led to some clear success, the nation needs to do 
far more to boost student learning. That is why the Common Core effort is so 
important: It builds on the standards movement by raising expectations, creating 
consistency, and supporting teachers who are looking to hone their craft. 

Like previous standards-based reforms, the Common Core arose from the shared 
desire of educators, policymakers, and other stakeholders to raise academic 
standards in schools. What is different about the Common Core is that multiple 
states are adopting the same standards and high-quality aligned assessments for 
their entire K- 1 2 education system. Local districts also are doing far more when it 
comes to adopting high-quality curricula and more focused capacity building. 20 

It is important to note that although standards-based reform does not have one con- 
sistent definition, the term implies several central components. 21 For one, the effort 
aims to focus schooling practices around a set of academic standards that specify 
what students should know and be able to do in each grade level and subject. 

For another, the academic standards should guide educators’ work in the class- 
room, helping teachers set goals for students. Advocates also believe that there 
should be assessments that are aligned to the standards, and that these stan- 
dards-based assessments should offer students an opportunity to demonstrate 
how well they have met the standards. 22 

This does not mean that standards-based reforms require teachers to teach in 
certain ways. Instead, educators — administrators, teachers, and support staff — 
have the freedom to develop learning experiences for students to attain grade-level 
standards. That autonomy is balanced with accountability. When schools fail 
to successfully prepare students for these tests, their states might take action in 
response. In the most extreme cases, the state might take over the school. 
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Critics of the Common Core and other standards-based reform efforts argue 
that the approach creates a one-size-fits-all education system that fosters teach- 
ing to the test. 23 And while these criticisms are misplaced, what is clear is that the 
Common Core will only be successful in improving student outcomes if it drives 
other positive changes in the education system. 


Previous research on standards-based reform 

When the federal government released the results of the latest National 
Assessment of Educational Progress test earlier this year, the results were surpris- 
ing. 24 From 2013 to 2015, outcomes dipped in some areas, and critics blamed the 
Common Core. 25 But it is too soon to gauge the success of the new standards. 

“Big change never happens overnight,” Secretary of Education Arne Duncan told 
reporters. “I’m confident that over the next decade, if we stay committed to this 
change, we will see historic improvements.” 26 

But one can get a better sense of the potential of the new Common Core stan- 
dards by looking at standards-based reform prior to the Common Core, and 
as noted above, part of our research effort was based on the work of Editorial 
Projects in Education’s Chris Swanson. In his original study from 2006, he 
found strong evidence for the power of standards-based reform. Swanson 
looked at NAEP data from 1997 to 2006 and showed that there is “a consistently 
positive relationship between achievement gains and the implementation of 
standards-based policies related to academic-content standards, aligned assess- 
ments, and accountability measures.” 27 

Other researchers have come to similar conclusions. Joshua Goodman, a 
researcher at Harvard University, found a positive effect in eighth-grade math- 
ematics when studying the effect of higher-quality academic standards on student 
achievement, though there were some inconsistent results. 28 Some years ago, the 
think tank Education Sector studied the impact of higher standards on the achieve- 
ment of a state’s lowest-performing students and found that states with higher stan- 
dards increased performance at higher rates than states with weaker standards. 29 

There is also significant evidence from other countries that standards can boost 
achievement. As recounted in Amanda Ripley’s book, The Smartest Kids in the World, 
Poland has seen its educational outcomes skyrocket in recent years, and today the 
Eastern European nation beats out Germany and the United Kingdom when it 
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comes to Program for International Student Assessment, or PISA, scores . 30 For 
Ripley, a key factor in the nation s turnaround was the fact that it raised its education 
standards and implemented a more rigorous and thoughtful instructional system. 

One critical piece of standards-based reform is how states hold districts and 
schools accountable for improving student performance. In an important study 
of the effects of state accountability systems on student outcomes, researchers 
Martin Carnoy and Susanna Loeb cataloged the strength of state accountability 
systems and found higher gains in NAEP in eighth-grade mathematics in high- 
accountability states . 31 Other key factors in standards-based reform include sup- 
port for teachers, along with additional dollars for low-income students . 32 

However, not all researchers interpret the impact of standards-based reform posi- 
tively. For example, researcher Laura Hamilton and her colleagues at the RAND 
Corporation have found the research to be inconclusive on the whole . 33 But over- 
all, the studies on standards-based reform seem to have found a positive impact. In 
a recent summary of the research, for instance, researcher Morgan Polikoff asked, 
“Does standards-based reform and accountability improve student outcomes ?” 34 
Polikoff concluded that “there is no reasonable way to read the literature and con- 
clude the answer is anything other than ‘yes .’” 35 
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CAP analysis 


In this study ; we looked at standards-based reforms over the past dozen years. 
Specifically we wanted to see if low-income students in states that had imple- 
mented standards-based reforms with greater fidelity performed better than others. 

First; we looked across the data to find patterns and trends, trying to connect policy 
developments to outcomes. It is this line of inquiry that led us to our first conclu- 
sion around certain states falling behind due to a lack of standards-based reform. 

As part of our work, we also modeled a regression analysis based on researcher 
Christopher Swanson’s previous study. In his report, Swanson looked at the 
impact of standards-based reform on National Assessment of Educational 
Progress scores, from 1 997 to 2006. 36 He focused on overall changes in policy 
and achievement from the mid-1990s to the mid-2000s. Our study looked only at 
the performance of low-income students on NAEP and goes up to 2013. Unlike 
Swanson, we focused on changes over smaller time intervals — every two years — 
since the passage of No Child Left Behind in 2002. 

As part of our regression analysis, we considered several indicators of standards- 
based reforms, such as “standards that are clear, specific, and grounded in con- 
tent,” whether state tests were “aligned to state standards,” and whether a “state 
sanctions low-performing schools.” We relied on the best data source of this 
kind — the Education Counts indicators developed by Education Week. 37 

For the regression, we created a standards-based reform score for each state. 

The score was based on the results of the state’s performance in three categories: 
standards; assessments; and accountability. Within each of these categories, there 
were multiple indicators relevant to that category. In the standards category, for 
instance, we looked at whether a state had grade-specific academic standards in 
English language arts, or ELA; mathematics; science; and social studies. 
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In the assessment category, there were indicators looking at numerous factors, 
including whether a state had aligned assessments on all three schooling levels — 
elementary, middle, and high school — and if the assessments included open-ended 
items on exams. The accountability category also included a number of indicators, 
including whether a state intervenes in low-performing schools and rewards high- 
performing schools. 

We rolled Education Week’s indicators of standards-based reform into a policy 
implementation score, similar to Swansons original analysis. If a state received 
credit on an indicator from Education Week, then the state had a higher standards- 
based policy reform score — all else being equal — than a state that did not receive 
credit on an indicator in our evaluation. On most policies, states were given 1 point 
for having each policy in place. So if a state’s math test aligned with the state’s math 
standards, it received 1 point, and if a state’s math test did not align with its math 
standards, it received 0 points. Some indicators counted for 2 points. 


Key indicators of Education Week's standards-based 
reform approach 

The indicators in our model fall into three categories, which are listed in the Appendix. 

The indicators fell into one of three categories. Standards indicators measured the 
quality of state content standards in different subjects. Assessment indicators rated 
whether state tests were aligned with state standards and gave states extra points 
for including more than just multiple-choice items. Finally, states received points 
for various aspects of their accountability systems, such as — “reward [ing] high- 
performing schools” and “sanction [ing] low-performing schools.” We normed the 
results within each category to a final 10-point scale and averaged them. 

To account for fiscal equity, we included an indicator about school funding, also 
from the Education Counts database. This measure expresses the difference 
between expenditures in the highest- and lowest-spending districts. The indicator 
is known as the “restricted range” because it ignores outlier districts. States with 
higher values for this indicator are more inequitable. All results are adjusted for dif- 
ferences in cost of living and student demographics. 

Then, we examined the relationship between the states’ policy scores over time 
and their progress on NAEP exams. We performed a regression analysis of these 
scores against the NAEP results of low-income students in each state. Our analysis 
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included these changes in policy and states’ overall policy scores. Note that since 
there were not the same number of indicators for each category some indicators 
had more weight than others. We provide more information about our regression 
analysis in the Appendix. 


Limitations 


Determining how well a state has implemented a set of education reforms over a 
dozen years is difficult, and there are significant variations and nuances in policy 
that we were unable to capture in any of our approaches. 

For this reason, we used both a qualitative and quantitative approach, looking to 
see trends in the data and trying to connect those trends to specific policy devel- 
opments in a state related to standards-based reform. 

And while the regression analysis is more empirical, it overlooks significant 
nuance. Education Week was unable to gather information about quality of 
reforms beyond whether a policy had been implemented. For instance, some 
state report cards, which contain some of the data upon which this analysis is 
based, are of higher quality than others. 

As noted above, variations in our model also can shift the outcomes. Without 
including the fiscal measure regarding the restricted range, for instance, policy 
changes generally have a weaker relationship with changes in achievement. 

It is also important to note that how Education Week measured some of the state pol- 
icies did undergo small adjustments over time. In 2007, Education Week adjusted the 
data, including the approach to state standards, changing from whether standards 
are “clear, specific, and grounded in content” to “course- or grade-specific.” 38 Also, 
after 2007, state standards policies were evaluated biannually rather than annually. 

Also, some categories had more indicators than others. For example, a states 
accountability policy score is based on four indicators, while the standards and 
assessments policy scores relied on five indicators, many with multiple sub indica- 
tors. Therefore, a single change in an accountability indicator would produce a 
greater change in the states overall score than a single change in a state’s standards 
policy due to the way that the category was constructed. Still, we believe these data 
are the most comprehensive available on state standards-based reform policies. 
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Results 


Our analysis of academic achievement over the past decade suggests that 
standards-based reform efforts have made a positive difference. We found that 
improvements in state standards-based policies were associated with academic 
growth for students from low-income families, as measured by the National 
Assessment of Educational Progress. 

Some of this evidence is anecdotal and impressionistic. We found, for instance, 
that states such as Kansas, Iowa, Idaho, Montana, and North and South Dakota 
showed some of the lowest gains on NAEP for low- income students since the late 
1990s, and broadly speaking, these states have been less committed to standards- 
based reform. 39 In contrast, many of the states that took a more robust standards- 
based approach to school reform — such as the District of Columbia, Tennessee, 
Massachusetts, and Florida — showed some of the largest gains. 

Using a more empirical approach — a regression analysis — we found that a 
1 -point increase in the standards-based policy implementation index was associ- 
ated with a 0.2-point gain on NAEP in fourth-grade math and a 0.4-point gain 
in eighth-grade reading. Our model controlled for states’ prior NAEP scores 
and past policy changes, as well as a measure of fiscal equity. We did not find any 
statistically significant relationships between improvements in standards-based 
policies and NAEP performance for low-income students in fourth-grade read- 
ing or eighth-grade math. 40 

One way to understand our regression findings is to look at specific examples. 
Take Iowa, for instance. The state has seen relatively limited overall gains on 
NAEP for low-income students, and we believe that one reason for this is that 
the state has not fully embraced standards-based reform. During the time period 
covered by our study, for example, Iowa’s standards-based reform policy scores 
were the lowest of any state in the United States. Likewise, the Hawkeye State 
did not show much improvement over time in its scores, and by 2013, the state’s 
policy scores were among the nation’s lowest. 
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Our findings are far from conclusive. For one, some of our work is plainly anec- 
dotal and thus potentially suffers from confirmation bias. Also, we have only 
shown a relationship between implementing standards-based reform policies 
and improvement on national assessments, and even there, we only found robust 
results on two of the four NAEP tests. 

Our study also did not capture all of the factors that could explain differences in 
academic performance in states, such as the quality of professional development 
or the level of school spending. And finally, our measure of policy implementation 
is limited and may not account for differences in quality of implementation. That 
being said, we believe that our findings — taken together — are valid. 
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Recommendations 


Embrace the Common Core 

Given the positive relationship between standards-based reforms and increased 
achievement, states should stick with the Common Core and its associated assess- 
ments. Unfortunately, some have decided to opt out of the Common Core, includ- 
ing Oklahoma and South Carolina. 41 What is notable is that these states have low 
achievement, and they score below the national average in most subject areas. 

For high-poverty students, Oklahoma and South Carolina do a bit better, scoring 
higher than the national average in some areas. But they still have a long way to 
go. In South Carolina, for instance, just 14 percent of low-income students are at 
grade level in middle school math. In Oklahoma, just 13 percent of low-income 
students are doing grade-level work in middle school math. 42 

Part of the benefit of the Common Core is that it raises expectations for all 
students. The standards are structured to build on top of one another and lead to 
college and career readiness. They are also internationally benchmarked against 
academic standards for high-achieving countries such as Korea and Singapore. 

The standards also emphasize critical thinking and problem solving — the very 
skills students need to be competitive in today’s workplace. 43 

Before the Common Core, the quality and expectations of state academic stan- 
dards varied greatly across the country. 44 Many states had weak standards, and 
Iowa’s English language arts standards earned an “F,” a failing grade, according to 
the Fordham Institute. 45 The Common Core standards are a major step forward 
in this regard, and the new standards were found to be “clearer and more rigorous 
than the ELA and math standards presently used by the vast majority of states.” 46 
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Embrace Common Core-aligned assessments and 
improve testing practices 


In our analysis, states received higher standards-based policy scores if they 
improved their testing practices. Specifically if a state added more extended- 
response items, which are more likely to assess higher-level thinking skills on the 
state tests, the state’s policy score would go up. At the same time, we also must 
acknowledge that some states may continue to use low-quality tests that do not 
cover all the standards or assess a student’s true ability to solve problems and 
understand concepts. Such low-level tests can lead to a narrowing of curricula 
and excessive test preparation. 

Generally, state assessments up to this point have been low quality, however, 
the new Common Core tests offer an opportunity to change that dynamic . 47 
Typically, the older assessments tested only the most basic knowledge and skills, 
were misaligned with what students were actually learning in school, and on 
some tests, students were not even asked to write or show their work. In short, 
assessments have long failed to make the grade . 48 

On the other hand, the Common Core-aligned assessments are different. They 
are designed to assess student thinking and problem-solving skills rather than 
memorization. Therefore, students will need to demonstrate their thinking, apply 
what they know, and write to be successful on these next-generation assessments. 
The Smarter Balanced assessments, for example, are computer adaptive, which 
means that the test gets easier or more challenging depending on student ability . 49 
In other words, the exam results would be far more detailed about what students 
actually know and are able to do. Simply put, by raising the bar for students, tests 
also have to be improved to accurately measure what students are learning. 


Build instructional capacity 

Standards-based reform does not happen in a vacuum, and states should support 
teachers to grow their practice as they transition to the new standards. Among 
other things, states can help support teachers’ professional growth by providing 
dedicated time for teachers to review and internalize the standards. Many teachers 
report not having enough time and capacity to delve deeply into the standards or 
to familiarize themselves with new problems or instructional items . 50 To solve this 
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problem, districts and schools could provide dedicated Common Core planning 
time, during which novice educators would work with an experienced teacher to 
create new lesson plans. Another option would be for schools and districts to pay 
for teachers to take summer courses or workshops in order to help them get a bet- 
ter sense of the new standards. 

States and districts also should create robust curricula, bringing the standards even 
closer to classroom practices. Between 2002 and 2009, for instance, Maryland 
supported its teachers in creating a common curriculum that could guide teachers’ 
work. 51 High-quality curricula also can provide teachers with lesson suggestions 
and instructional materials that can substantially reduce teacher workload as states 
transition to new standards and associated assessments. Individual districts, such 
as Washoe County, Nevada, also have supported teachers in creating instructional 
materials aligned with the Common Core and found ways to give teachers more 
time to work with each other to develop these standards. 52 But these stories of sus- 
tained and thorough commitment to the Common Core are the exception rather 
than the rule. 
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Conclusion 


Should states continue to use the Common Core standards as a lever for improv- 
ing outcomes for low- income students? Our research provides an answer, and 
the evidence on standards-based reform strongly supports the potential of the 
Common Core to drive improvements in educational outcomes for all stu- 
dents. Our study adds to this body of research, suggesting that states that have 
embraced standards-driven reforms over the past decade have enabled low- 
income students to perform at much higher levels. 

However, a commitment to new standards and higher- quality assessments will 
not be enough. States also must build capacity to support educators to rise to the 
challenges of teaching to the Common Core. Bringing the standards fully to life 
and achieving their potential requires teachers to master the standards and learn 
new techniques. States can help make these efforts successful by creating new 
opportunities for teachers to collaborate and providing new curricular materials 
that can guide teachers’ work. But it all starts with setting high academic stan- 
dards that support and guide best practice. 
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Appendix: Methodology 53 


For this report, we followed the analytic approach taken by Christopher Swanson, 
vice president of research and development at the Editorial Projects in Education, 
for his study of standards-based reform and student achievement. 54 Our findings 
are based on regressions that compared changes in achievement for low-income 
students with changes in policy. We used a robust fixed-effects model with 
states as the unit of analysis. We regressed differences in National Assessment 
of Educational Progress scores for low-income students on measures of their 
standards-based reform policies and their previous NAEP scores. 

To measure student achievement, we used average scores from the NAEP grade- 
level tests in math and reading. We analyzed changes in test scores between each 
NAEP administration from 2003 through 2013; namely, 2003-2005, 2005-2007, 
2007-2009, 2009-2011, and 2011-2013. 

We measured policy differences across states using information from the 
Education Counts database. We created a policy implementation score that 
includes several Education Counts indicators that measure various aspects of state 
standards, assessments, and accountability systems. A full list of the indicators is 
in Table A1 below, and Education Week created the categories within which the 
indicators were placed. The standards category indicators measure the quality of 
state standards — for example, are they clear and specific? The assessments cat- 
egory indicators primarily track the alignment of state tests with state standards — 
for example, does the test measure what is included in state standards? Finally, the 
accountability category items include information about public reporting and also 
“rewards” and “sanctions” for schools based on their performance. We included a 
different subset of the accountability indicators than Swanson. 

Education Counts indicators developed by Education Week assign points to states 
that have certain policies in place. 55 Most indicators only counted for 1 point. 

We note exceptions in Table A1 below. The scores are added up in each major 
category and then converted to a scale of 10 total points. We focused on two-year 
internals, shorter time periods than Swanson. 
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We also included an indicator about school funding equity, also from the 
Education Counts database. This measure expresses the difference between 
expenditures in the highest- and lowest- spending districts. The indicator is known 
as the “restricted range” because it ignores outlier districts. States with higher val- 
ues for this indicator are more inequitable. All results are adjusted for differences 
in cost of living and student demographics. 


We illustrate our model in the text box below. We regressed the change in NAEP 
scores over a given two-year period onto policy changes from the same two-year 
period, controlling for where they started. For example, we regressed changes in 
fourth-grade reading scores — 2009 to 20 1 1 — on changes in their policy imple- 
mentation scores — 2009 to 2011 — controlling for their 2009 NAEP score, their 
2009 policy score, and their 2011 funding equity indicator. 


Change in NAEP fourth-grade 
reading score (from yearT-1 toT) is 
related to change in policy imple- 
mentation score (T-1 toT) + NAEP 
fourth-grade reading baseline (T-1) 
+ policy implementation baseline 
(T-1 ) + restricted range (T). 


Our model could not account for all the factors that could affect student 
achievement on the NAEP tests. For example, there are factors outside schools’ 
control that might have affected the performance of low-income students. 
Nevertheless, because we used a fixed-effects model at the state level, we were 
able to partially account for differences across states. The coefficients from our 
regression analysis are shown below. 


TABLE A1 

Results from regression of NAEP scores on 
standards-based policy measures 



Math 

Reading 

Grade 4 

Grade 8 

Grade 4 

Grade 8 

Change in policy implementation score 

0.23* 

-0.049 

-0.19 

0.42* 

Prior policy score 

0.19 

-0.12 

-0.23 

0.34 

Prior achievement on NAEP 

-0.45** 

-0.34** 

-0.052** 

-0.27** 

Restricted range 

0.000 

0.000 

0.000 

0.000 


* Effect is significant at the p = 0.1 0 level. 

** Effect is significant at the p = 0.05 level. 

Note: We do not report the regression constant. 

Sources: Authors' analysis based on Education Week policy indicators and NAEP state results, respectively, from Education Counts Research 
Center, "Select Indicators," available at http://www.edcounts.org/createtable/stepl .php (last accessed November 201 5); National Center for 
Education Statistics, "NAEP Data Explorer," available at http://nces.ed.gov/nationsreportcard/naepdata/dataset.aspx (last accessed November 
2015). 
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List of indicators 


Standards 

1. Core Standards: 

• State has standards in the core subjects 

2 = all subjects, 1 = some subjects, 0 = no subjects 

2. Content-based English language arts, or ELA, standards: 

• State has standards that are clear, specific, and grounded in content in 
English/language arts in elementary school 

• State has standards that are clear, specific, and grounded in content in 
English/language arts in middle school 

• State has standards that are clear, specific, and grounded in content in 
English/language arts in high school 

2 = all grade levels, 1 = some grade levels, 0 = no grade levels 

3. Content-based math standards: 

• State has standards that are clear, specific, and grounded in content in math- 
ematics in elementary school 

• State has standards that are clear, specific, and grounded in content in math- 
ematics in middle school 

• State has standards that are clear, specific, and grounded in content in math- 
ematics in high school 

2 = all grade levels, 1 = some grade levels, 0 = no grade levels 

4. Content-based science standards: 

• State has standards that are clear, specific, and grounded in content in science 
in elementary school 

• State has standards that are clear, specific, and grounded in content in science 
in middle school 

• State has standards that are clear, specific, and grounded in content in science 
in high school 

2 = all grade levels, 1 = some grade levels, 0 = no grade levels 
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5. Content-based social studies standards: 

• State has standards that are clear, specific, and grounded in content in social 
studies in elementary school 

• State has standards that are clear, specific, and grounded in content in social 
studies in middle school 

• State has standards that are clear, specific, and grounded in content in social 
studies in high school 

2 = all grade levels, 1 = some grade levels, 0 = no grade levels 


Assessments 

6. State assessments include short answer and extended response items 

• State elementary- school test includes short answer test items 

• State middle- school test includes short answer test items 

• State high-school test includes short answer test items 

• State elementary-school test includes extended response test items in 
English/writing 

• State middle school test includes extended response test items in 
English/ writing 

• State high-school test includes extended response test items in English/writing 

• State elementary- school test includes extended response test items in sub- 
jects other than English/writing 

• State middle-school test includes extended response test items in subjects 
other than English/ writing 

• State high-school test includes extended response test items in subjects other 
than English/writing 

2 = Short answer AND extended response, 1 = short answer OR 
extended response, 0 = multiple- choice only or no assessment 

7. State ELA test is aligned to state standards 

• Elementary-school ELA test is aligned to state standards 

• Middle-school ELA test is aligned to state standards 

• High- school ELA test is aligned to state standards 

1 = any or all grade levels, 0 = no grade levels 
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8. State mathematics test is aligned to state standards 

• Elementary school mathematics test is aligned to state standards 

• Middle-school mathematics test is aligned to state standards 

• High- school mathematics test is aligned to state standards 

1 = any or all grade levels, 0 = no grade levels 

9. State science test is aligned to state standards 

• Elementary-school science test is aligned to state standards 

• Middle-school science test is aligned to state standards 

• High- school science test is aligned to state standards 

1 = any or all grade levels, 0 = no grade levels 

1 0. State social studies test is aligned to state standards 

• Elementary-school social studies/history test is aligned to state standards 

• Middle-school social studies/history test is aligned to state standards 

• High-school social studies/history test is aligned to state standards 

1 = any or all grade levels, 0 = no grade levels 


Accountability 

1 1 . State has school report cards 

1 = yes, 0 = no 

1 2. State sanctions low-performing schools 

1 = yes, 0 = no 

13. State rewards high-performing schools 

1 = yes, 0 = no 

14. State requires exit exams 

1 = yes, 0 = no 

^Subjects were counted as ‘yes’ if they had indicator in place in any grade level. 


Source: List of indicators was retrieved from Education Counts Research Center, "Select Indicators," available at http://www.edcounts.org/ 
createtable/stepl .php (last accessed November 201 5). 
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